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Abstract. We study branching laws for a classical group G and a symmetric subgroup 
H. Our approach is through the branching algebra, the algebra of covariants for H in the 
regular functions on the natural torus bundle over the flag manifold for G. We give concrete 
descriptions of certain subalgebras of the branching algebra using classical invariant theory. 
In this context, it turns out that the ten classes of classical symmetric pairs (G, H) are 
associated in pairs, (G, H) and {H' , G'), and that the (partial) branching algebra for (G, H) 
also describes a branching law from H' to G'. (However, the second branching law may 
involve certain infinite-dimensional highest weight modules for H'.) To highlight the fact 
that these algebras describe two branching laws simultaneously, we call them reciprocity 
algebras. Our description of the reciprocity algebras reveals that they all are related to 
the tensor product algebra for Gi„. This relation is especially strong in the stable range. 
We provide explicit descriptions of reciprocity algebras in the stable range in terms of the 
tensor product algebra for GL„. This is the structure lying behind formulas for branching 
multiplicities in terms of Littlewood-Richardson coeSicients. 



1. Introduction 

One of the basic problems in the study of transformation groups and finite dimensional 
representation theory is the branching problem: describing how irreducible representations of 
a compact group (mutatis mutandis , a reductive complex algebraic group) decompose when 
restricted to a subgroup. More specifically, given a group G and a subgroup H, we may 
consider the branching problem for the pair {G,H). This paper introduces and establishes 
some basic features of an approach to the branching problem for a class of particularly 
interesting cases: the classical symmetric pairs, in which the group G is a classical group 
and the subgroup if is a symmetric subgroup - the fixed points of an involution of G. One 
can organize these pairs in 10 families, listed in table I (see page 19). 

There is already a substantial literature on branching rules for classical symmetric pairs 

H, 0, 0, i, i, i, cDi, cn, m, m, en, m, m, Ea, m, eh, m, m, m, eh, 
m, m, Ea, m, eh, m, m, m, m, m, m, m, m, m, m, 

EH, E3, 

inni, Ei- 

A substantial portion of this work is combinatorial in nature, and is formulated in terms 
of tableaux and Littlewood-Richardson coefficients. This approach began with the work of 
Littlewood and Richardson jlH] on tensor products of representations of GLn and Little- 
wood's Restriction Formula jSU] [IH] for restriction of certain representations of GLn to the 
orthogonal group or the symplectic group. More recent work in this tradition includes 
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several papers by Koike and Terada [SI], [SE]; |SZ| and by R.W. King and collaborators 

m, m, m, m, i, eh, m, m, m- 

A group G embedded diagonally in the product GxG is a symmetric subgroup, correspond- 
ing to the involution which exchanges the two copies of G. An irreducible representation 
of G X G is just a tensor product pi 55 P2 of irreducible representations of G. Thus, the 
branching problem for symmetric pairs includes the tensor product problem - to describe 
how a tensor product of representations of a group G decomposes into irreducible pieces. In 
the 1990s, striking progress on the tensor product problem came from an unexpected source 
- the theory of quantum groups (see [Tl l^UH^ ). 

Quantum groups are one-parameter deformations of the universal enveloping algebra of a 
semisimple Lie algebra. The extra parameter was used by Lusztig ([^) and Kashiwara (523|) 
to define bases ("canonical bases" or "crystal bases") for these algebras. The properties of 
these bases permitted combinatorial descriptions of tensor product multiplicities, providing 
an extension of the Littlewood- Richardson Rule for GLn to all simple groups. This approach 
has been elucidated and simplified by the path model of Littelmann (see |^^47j ). which also 
provides branching rules for restriction from a group G to the Levi component of a parabolic 
subgroup. 

This paper takes another approach to the branching problem for classical symmetric pairs. 
Its main tool is invariant theory, represented primarily by the geometric version of highest 
weight theory (see jTU], 0) and by classical invariant theory [TniElE^- (At the 

moment, parts of our development also depend on the combinatorial theory of the Littlewood- 
Richardson Rule. However, our approach also suggests alternative approaches to these re- 
sults, and we hope eventually to obtain independent proofs.) To describe the branching rule 
for a classical symmetric pair {G,H), we study a naturally defined algebra B{G,H), which 
we call the branching algebra for the pair {G, H). The algebra B{G, H) carries a multigrad- 
ing by Aq x A^, the semigroups of dominant weights of G and H. The dimension of the 
homogeneous component B{G, H)^^'"^^ labeled by {(f), ip) G Aq x A'^ equals the multiplicity 
of in p'^, where p'^ (respectively, p"^) is the irreducible representation of G with highest 
weight (p (respectively, irreducible representation of H, with highest weight ip). Branching 
algebras were studied in jUU], but have been mostly neglected since. 

In addition to the full branching algebras, our theory distinguishes certain families of 
graded subalgebras, depending on integer parameters. For a certain range of the parameters, 
the subalgebras have a particularly simple structure. We call this the stable range, and the 
resulting algebras, stable branching algebras. 

This approach must of course give the same numbers as are produced by the combinato- 
rial, quantum group and path model methods. We have already shown in ^Hl how to use 
branching algebras systematically to express stable branching multiplicities for all families 
of classical symmetric pairs in terms of Littlewood- Richardson coefficients. 

We feel that branching algebras have several virtues in addition to providing an efficient 
route to formulas for branching multiplicities. One is, they provide finer information than 
multiplicities: they provide highest weight vectors. Indeed, the homogeneous component 
B{G, HY'f''^^ of B{G, H) consists of the V^-highest weight vectors for if in a specific realization 
of the representation p**^ of G. Thus, if one understands B{G, H) well enough, one gets 
information about how //-modules sit inside G-modules. For the classical symmetric pairs, 
this possibility can in fact be realized: in [T7j, the authors have shown how to construct an 

2 



explicit basis for the GLn tensor product algebra. In further work, Howe and Lee ^3] have 
extended this construction to apply to most of the families of branching algebras. 

A second positive feature of branching algebras is, they collect all multiplicity information 
into one coherent structure. This has several potential implications. For example, it has 
been observed in many cases that branching multiplicities can be interpreted as the number 
of integral points in certain convex polytopes. The theory of SAGBI bases (jSH]) provides a 
strategy for attaching lattices cones to affine algebras. The explicit bases mentioned above 
permit one to realize the SAGBI strategy effectively. The insight gained into the structure 
of branching algebras allows one in turn to predict the existence of convex polytopes whose 
integral points define the desired multiplicities [21, 0, 113 • 

The algebra structure also provides natural connections between the different branching 
algebras. The stable branching algebras for all 10 families of classical symmetric pairs are 
closely related to each other, and in particular, they all can be described in terms of the 
GLn branching algebra. These relationships lie behind the various formulas expressing stable 
multiplicities for all families in terms of Littlewood- Richardson coefficients jlbj . 

These connections in fact go beyond branching algebras. By some remarkable coincidences, 
there are also very close relationships between branching algebras for classical symmetric 
pairs and classical Kostant-Rallis actions (see [10], [HZl, [HH])- By a Kostant-Rallis action, 
we mean the complexified action of if on g/f), where if is a maximal compact subgroup of a 
semisimple Lie group G, and g and () are the Lie algebras of G and H respectively. Some key 
results about these actions were given in [lUj. but some aspects of their structure are still not 
completely settled. The tight connection of classical Kostant-Rallis actions with branching 
algebras can be used to improve understanding of these important examples. Beyond these 
immediate applications, branching algebras should provide a useful general tool for analyzing 
group actions. 

We also feel that the branching algebra approach sheds light on some of the basic con- 
structions of the combinatorial theory. For example, the combinatorial description tensor 
product multiplicities for GLn by means of Littlewood-Richardson tableaux still has an air 
of mystery about it. A number of experts have remarked on the curious circumstance that, 
although tensor product multiplicities are obviously symmetric in the two tensor factors, 
the Littlewood-Richardson description of multiplicities is not symmetric. The SAGBI basis 
description of the GLn tensor product algebra [T7] provides a natural interpretation of the 
information encoded by a Littlewood-Richardson tableau, including providing a straightfor- 
ward context for the asymmetrical treatment of the two factors. 

The efficient packaging of all multiplicity formulas for a given pair in one algebraic structure 
also seems to provide an efficient way of computing explicit results in low rank examples. 
The authors hope to treat a collection of explicit cases in a future publication. 

Finally, although the branching algebra construction is not limited to symmetric pairs, it 
has a remarkable additional feature in the case of classical symmetric pairs. As explained in 
\n\ |12j . classical invariant theory gives rise to natural correspondences between representa- 
tions of pairs ( "dual pairs" ) of classical groups. It turns out that these correspondences can 
be refined to correspondences between branching laws for symmetric pairs. The 10 families 
of classical symmetric pairs pair up into 5 families of pairs of pairs (see Table II on page 19), 
in such a way that a branching algebra for one pair can also be interpreted as a branching 
algebra for the corresponding pair. Thus, one branching algebra simultaneously describes 
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branching between representations of two classical symmetric pairs. This can be thought of 
as a reciprocity law, analogous to Frobenius Reciprocity in the theory of group representa- 
tions. Hence we call these branching algebras reciprocity algebras. (Multiplicity versions of 
these reciprocity laws are implicitly included in the general result of [H]). We should also 
remark that in most cases, the relevant representations for some of the groups involved in the 
pairs will be infinite dimensional. However, the infinite dimensional representations which 
occur are of a very special sort, and share many of the characteristics of finite-dimensional 
representations.) 

For all the above reasons, we hope that branching algebras will serve as a useful comple- 
ment to the already developed approaches to describing branching laws. 

1.1. Overview. We will give the construction of a branching algebra in the following sec- 
tion. We shall illustrate how the algebra structure emphasizes the coherence of the branch- 
ing problem across representations of G, rather than simply providing a representation-by- 
representation answer. Further, the algebra structure goes beyond the numerical nature of 
multiplicities, and is the basis of some of the well-known multiplicity results. 
There are three main parts to this paper: 

(a) Background: preliminary knowledge on multiplicity-free spaces, the theory of dual 
pairs and the classification of classical symmetric pairs, leading to the concept of 
reciprocity algebra. One can group the classical symmetric pairs into 10 families, 
and it turns out that these families are associated in pairs, (G, H) and {H', G') (see 
Tables I and II), and that the branching algebra for [G, H) also describes a branching 
law from H' to G'. Hence, we call the algebra that describes the two related branching 
laws a reciprocity algebra. 

(b) Reciprocity: explaining the underpinnings of the reciprocity of multiplicities for 
the 5 pairs of reciprocity algebras, with detailed discussions for certain representative 
pairs. 

(c) Stability: describing the branching algebra for a symmetric pair (G, H) in terms of 
a GLn tensor product algebra. This emphasizes the underlying importance of the 
GLn branching algebras, since their structure relate to all others. This phenomenon 
is also the basis for the expression of multiplicity formulas for representations of 
classical groups in terms of Littlewood- Richardson coefficients - a recurring theme in 
the literature. The present paper brings more structure to these multiplicity results. 

The first four sections provide the necessary background: 

§3: We provide the notations for the parametrization of representations in §3.1. Next, 
we discuss some preliminary concepts on multiplicity-free spaces and prove a main 
theorem (see Theorem 3.3) on embedding the associated graded algebra of a graded 
G-multiplicity-free space into the algebra TZ{G/Ug)- The final subsection discusses 
the rudiments of classical invariant theory, formulated in terms of the theory of dual 
pairs. 

§4: Section 4 begins with the simplest example of a pair of branching algebras for GLn, 
leading us to the concept of reciprocity algebras. This is where we bring forth the 
classification of classical symmetric pairs and the theory of dual pairs to introduce 
the 5 pairs (also called see-saw pairs) of reciprocity algebras. 

For the second part of the paper, we organize discussions of the see-saw pairs as follows: 
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§5: discusses the branching from GLn to 0„ (note that branching from GL2n to Sp2n can 
be treated similarly); 

§6: discusses the tensor product algebra for 0„ (note that the tensor product algebra of 

Sp2n can be treated similarly); 
§7: provides a more general treatment of the reciprocity for {GLn,GLm)- 

For the sake of brevity, we will not discuss the branching from Sp2{m+n) to Sp2m x Sp2n, 02n 
to GLn and Sp2n to GL„. They can be treated similarly. 

In the final part of this paper we begin with some important comments on stability results 
for branching in §8. We demonstrate the theoretical underpinnings for stability results, 
highlighting certain specific see-saw pairs: 

§9: we interpret the associated graded of the branching algebra from GLn to 0„ as a 
(0, l)-subalgebra (see Definition 3.1) of the tensor product algebra of GLm in the 
stable range n > 2m. 

§10: we interpret the associated graded of the On tensor product algebra as a (0, 1)- 
subalgebra of a triple product of tensor product algebras of GLn, GL^ and GLi in 
the stable range n > 2(m + ^). 

§11: we interpret the branching algebra of On+m to 0„ x Om as a (0, l)-subalgebra of a 
triple tensor product algebra of GLi in the stable range min (n, m) > 2i. 

In all these cases, we show that the branching algebras associated to symmetric pairs can 
all be identified with suitable branching algebras associated to the general linear groups. 
Thus, if we can have control of the solution in the general linear group case, we will have 
some control of the other classical groups. And indeed, we do (see ^7\). The other non- 
trivial examples will be important extensions of this work, and we hope to see them in further 
papers, for example, jT3j. 

Acknowledgements: We thank Kenji Ueno and Chen-Bo Zhu for discussions on the proof 
in the Appendix. The second named author also acknowledges the support of NUS grant 
R-146-000-050-112. The third named author was supported by NSA Grant # H98230-05-1- 
0078. 

2. Branching Algebras 

For a reductive complex linear algebraic G, let Ug be a maximal unipotent subgroup of 
G. The group Uq is determined up to conjugacy in G [5j. Let Aq denote a maximal torus 
which normalizes Ug, so that Bg = Ag ■ Ug is a Borel subgroup of G. Also let Aq be the 
set of dominant characters of Ag - the semigroup of highest weights of representations of 
G. It is well-known jS] J3] and may be thought of as a geometric version of the theory of 
the highest weight, that the space of regular functions on the coset space G/Ug, denoted by 
TUG/Ug), decomposes (under the action of G by left translations) as a direct sum of one 
copy of each irreducible representation (with highest weight ip) of G (see plj): 

nG/UG) ^ V^. (2.1) 

We note that 1Z{G/Ug) has the structure of an A^-graded algebra, for which the are the 
graded components. To be specific, we note that since Ag normalizes Ug, it acts on G/Ug 
by right translations, and this action commutes with the action of G by left translations. 
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The following result is well-known, probably folklore. We provide the proof anyway, since 
we are not able to find a suitable reference. 

Proposition 2.1. The algebra of regular functions TZ{G/Ug) is an A^-graded algebra, under 
the right action of Aq- More precisely, the decomposition (2.1) is the graded algebra decompo- 
sition under Aq, where is the Ac-eigenspace corresponding to (p & Aq with = w*{tp~^). 
Here w is the longest element of the Weyl group with respect to the root system determined 
by the Borel subgroup Bq- 

Proof. Since Aq is commutative and reductive, we can decompose TZiG/UG) into eigenspaces 
for the right action of Aq. These eigenspaces must be invariant under the action of G 
by left translations. Since is irreducible for the action of G, it must belong to a sin- 
gle Aq eigenspace. Let / be the highest weight vector for Eg in V^i,. Then by definition 
La{f) = '4'{o,){f), where a e Aq and Lg indicates left translation by g: Lg{f){h) = f{g^^h) 
for any h in G, representing a point in G/Uq- If w is the longest element of the Weyl group, 
then the Bq orbit BqwUg is dense in G/Ugi so that / is determined by its restriction to 
BgwUg- We compute that 

il^{a)f{w) = La{f){w) = f{a'''^w) = f{ww~'^a~^w) = Rw-^a-^wf{w), 

where Rgf{h) — f{hg) refers to the right translation of / by g. If belongs to the 
eigenspace for the right action of Ag, then this equation implies that 

jjj(a) ^(l)(w-^a-^w), or ijj ^ w* {(f)-^) , (2.2) 

where w* indicates the action of w on Ag resulting from conjugation of Aq by w. Thus, 
the are exactly the eigenspaces for the right action of Ag, with the Ac-eigencharacter 
related to the highest weight by the equation (2.2). Since the right action by Ag (as well as 
the left action by G) is an action by algebra automorphisms of TZ{G/Ug), it is easy to check 
that if /i is a 0-eigenfunction for Ag, and /2 is a ^^-eigenfunction, then the product /1/2 is 
a 0^-eigenfunction for Aq. It follows that 

so that, indeed, the decomposition (2.1) defines a structure of an yl^-graded algebra on 

n{G/UG). □ 

Now let i7 C G be a reductive subgroup, and let Uh be a maximal unipotent subgroup 
of H. We consider the algebra TZ{G/Ug)^" , of functions on G/Ug which are invariant 
under left translations by Uh- Let Ah be a maximal torus of H normalizing Uh, so that 
Bh '■= Ah ■ Uh is a Borel subgroup of H. Then TZ{G /Ug)^" will be invariant under the 
(left) action of Ah- and we may decompose TZ{G/Ug)^" into eigenspaces for Ah- Since 
the functions in 1Z{G /Ug)^" are by definition (left) invariant under Uh, the (left) Ah- 
eigenfunctions will in fact be (left) Bh eigenfunctions. In other words, they are highest 
weight vectors for H. Hence, the characters of Ah acting on (the left of) TZ{G /Ug)^" will 
all be dominant with respect to Bh, and we may write 71{G/Ug)^" as a sum of (left) Ah 
eigenspaces {TZ{G /Ug)^")^ for dominant characters x of H: 

n{G/UGf- = {n{G/UGfn''- (2-3) 
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Since the spaces V^p of decomposition (2.1) are (left) G-invariant, they are a fortiori 
left if-invariant, so we have a decomposition of TZ{G/Ug)^" into right A^-eigenspaces 



i-4> • 



nG/Ucf" = niG/Ucf" n := niG/Uc) 

Combining this decomposition with the decomposition (2.3), we may write 

nG/UGr"= inG/ucf/r. (2.4) 

To emphasize the key features of this algebra, we note the resulting consequences of 
decomposition (2.4) in the following proposition. 

Proposition 2.2. (a) The decomposition (2.4) is an (A'^ x Ajj)-graded algebra decom- 
position ofn{G/UGY". 
(b) The suhspaces {71{G/Ug)'^")^ tell us the x highest weight vectors for Bh in the 
irreducible representation of G. Therefore, the decomposition 

nG/Uaf/ = (7^(G/f/c)^-)'^ 

tells us how decomposes as a H -module. 

Thus, knowledge olTZ{G /Ug)^" as a (A^ x y4^)-graded algebra tell us how representations 
of G decompose when restricted to if, in other words, it describes the branching rule from 
G to H. We will call niG/Uc)^" the (G, H) branching algebra. When G - H x H, and H 
is embedded diagonally in G, the branching algebra describes the decomposition of tensor 
products of representations of H, and we then call it the tensor product algebra for H. More 
generally, we would like to understand the {G, H) branching algebras for symmetric pairs 
{G,H). 



3. Preliminaries and Notations 

3.1. Parametrization of Representations. Let G be a classical reductive algebraic group 
over C: G = GL,„(C) = the general linear group; or G = 0„(C) = 0„, the orthogonal 
group; or G = S'p2n(C) = Sp2n, the symplectic group. We shall explain our notations 
on irreducible representations of G using integer partitions. In each of these cases, we 
select a Borel subalgebra of the classical Lie algebra and coordinatize it, as is done in jlOj . 
Consequently, all highest weights are parameterized in the standard way (see jTU]). 

A non-negative integer partition A, with k parts, is an integer sequence Ai > A2 > . . . > 
Afc > 0. We may sometimes refer to A as a Young or Ferrers diagram. We use the same 
notation for partitions as is done in jH^I- For example, we write i{X) to denote the length (or 
depth) of a partition, i.e., i{X) = k for the above partition. Also let |A| = J2i be the size 
of a partition and A' denote the transpose (or conjugate) of A (i.e., (A')j = \{Xj : \j > i}\). 

GLji Representations: Given non-negative integers p, q and n such that n > p + q and 
non-negative integer partitions A"*" and A~ with p and q parts respectively, let F^\^ denote 
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the irreducible rational representation of GLn with highest weight given by the n-tuple: 
(A+, A ) = [Xl, A^, ■ ■ ■ , Ap , 0, ■ ■ ■ ,0, — Ag , • ■ ■ , —Xi ) 

n 

If A" = (0) then we will write ^(^"J for F^^^'^'l Note that if A+ = (0) then [P^n))* is 

equivalent to i^^^''"^- More generally, ^^^^^^1^^* *° ^V''"^' 

On Representations: The complex orthogonal group has two connected components. Be- 
cause the group is disconnected we cannot index irreducible representations by highest 
weights. There is however an analog of Schur-Weyl duality for the case of 0„ in which 
each irreducible rational representation is indexed uniquely by a non-negative integer parti- 
tion u such that (z/')i + (z/')2 < n. That is, the sum of the first two columns of the Young 
diagram of u is at most n. We will call such a diagram 0„-admissible (see Chapter 10 
for details). Let E^^-^ denote the irreducible representation of On indexed z/ in this way. 

An irreducible rational representation of SOn may be indexed by its highest weight. In 
[TU] Section 5.2.2, the irreducible representations of 0„ are determined in terms of their 
restrictions to SOn (which is a normal subgroup having index 2). We note that if £(z/) 7^ |, 
then the restriction of E'j^^-^ to SOn is irreducible. If £(z/) = | (n even), then E^^s^ decomposes 
into exactly two irreducible representations of SOn- See jTU] Section 10.2.4 and 10.2.5 for the 
correspondence between this parametrization and the above parametrization by partitions. 

The determinant defines an (irreducible) one- dimensional representation of 0„. This repre- 
sentation is indexed by the length n partition = (1, 1, ■ ■ • , 1). An irreducible representation 
of On will remain irreducible when tensored by E^^^, but the resulting representation may 
be inequivalent to the initial representation. We say that a pair of 0„-admissible partitions 
a and /? are associate if E^^-^ E'^^^ = E^^-^ . It turns out that a and (3 are associate exactly 
when (a')i + (/5')i = n and (a')i = (/?')i for all i > 1. This relation is clearly symmetric, and 
is related to the structure of the underlying SO^-representations. Indeed, when restricted 
to SOn-, E^^-^ = E^^-^ if and only if a and (3 are either associate or equal. 

Sp2n Representations: For a non-negative integer partition v with p parts where p < n, let 
^(2n) denote the irreducible rational representation of Sp2n where the highest weight indexed 
by the partition u is given by the n tuple: 

(z/i,z/2,--- ,i/p,0,--- ,0) 

n 



3.2. Multiplicity-Free Actions. Let G be a complex reductive algebraic group acting on 
a complex vector space V- We say is a multiplicity-free action if the algebra V{V) of 
polynomial functions on V is multiplicity free as a G module. The criterion of Servedio- 
Vinberg [HUJ says that V is multiplicity free if and only if a Borel subgroup -B of G has a 
Zariski open orbit in V- In other words, B (and hence G) acts prehomogeneously on V (see 
|59j). A direct consequence is that B eigenf unctions in V{V) have a very simple structure. 
Let G ViV) he a B eigenfunction with eigencharacter ip, normalized so that Q^p{vo) = 1 
for some fixed Vq in a Zariski open B orbit in V. Then is completely determined by ip: 
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For V = b ^Vq in the Zariski open B orbit, 

Q4^) = Q4b~^vo) = tp{b)Q4vo) = tp{b), b e B. 

is then determined on all of V by continuity. Since B = AU, and U = {B, B) is the 
commutator subgroup of 5, we can identify a character of B with a character of A. Thus 
the B eigenfunctions are precisely the G highest weight vectors (with respect to B) 'mV{y). 
Further 

and so the set of A'^(y) = j^/' G \ ^ 0} forms a sub-semigroup of the cone A'^ of 
dominant weights of A. 

An element 1) of a semigroup is primitive if it is not expressible as a non-trivial 
product of two elements of the semigroup. The algebra ViV)^ has unique factorization 
(see [13 ). The eigenfunctions associated to the primitive elements of y4"'"(V^) are prime 
polynomials, and V{V)^ is the polynomial ring on these eigenfunctions. li ip = ipiip2) then 
Qxl) = QipiQ'<p2- Thus, if ijj is not primitive, then the polynomial cannot be prime. An 
element 

^ = n.tiV'? 

has Cj's uniquely determined, and hence the prime factorization 

Consider a multiplicity-free action of G on an algebra W. In the general situation, we 
would like to associate this algebra W with a subalgebra of 71{G/U). With this goal in 
mind, we introduce the following notion: 

Definition 3.1. Let V = ^xeA+ denote an algebra graded by an abelian semigroup A'^ . 
IfW(^Visa subalgebra ofV, then we say that W is a (0, l)-subalgebra of V if 

\ez 

where Z is a sub-semigroup of A~^ , which we will denote by A~^(W) = Z. Note that W is 
graded by A+(W). 

In what is to follow, we will usually have V = V{V) (polynomial functions on a vector 
space V) and A'^ will denote the dominant chamber of the character group of a maximal 
torus A of a reductive group G acting on V. In this situation, we introduce an ^"'"-filtration 
on W as follows: 

cj><-ip 

where the ordering < is the ordering on A'^ given by (see j55j ) 

i'l < V'2 if is expressible product of 

rational powers of positive roots. 

Note that positive roots are weights of the adjoint representation of G on its Lie algebra g. 
We refer to the abelian group structure on the integral weights multiplicatively. Also, it will 
turn out that we only need positive integer powers of the positive roots. 
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Next consider the more specific situation where W which is a G-invariant and G-multiphcity- 
free subalgebra of a polynomial algebra ViV). Suppose that has unique factorization. 
Then is a polynomial ring and yl"'"(W) is a free sub-semigroup in generated by 
the highest weights corresponding to the non-zero graded components of W. Write the G 
decomposition as follows: 

noting that is an irreducible G module with highest weight ip. 

If 5 occurs with positive multiplicity in the tensor product decomposition 

® = dim HomciWs, ® W^) W^, 

5 

then 5 < (ptp. If 

Wr, C W^"^^ and C W^^\ i.e., r/ < and 7 < V', 
then it follows that 

Thus 

y\;i<t>) . yyCV') ^ )/y(</'V')_ 
We have now an A^-filtered algebra 

W= [j W^^\ 

and this filtration is known as the dominance filtration 1^3] • 

With a filtered algebra, we can form its associated algebra which is graded: 

where 

{Gi^+Wf = W^^^/ (0 W^-^M . 

Theorem 3.2. Consider a multiplicity-free G-module W with a -filtered algebra structure 
such that W is a unique factorization domain. Assume that the zero degree subspace of W 
is C. Then there is a canonical -graded algebra injection: 

Gr^+TT : Gr^+W ^n{G/U). 

Proof. In ^Ij it is shown that under the above hypothesis, is a polynomial ring on 
a canonical set of generators. Now, is a y4+-graded algebra and therefore, there exists 
an injective y4"'"-graded algebra homomorphism obtained by sending each generator of the 
domain to a (indeed any) highest weight vector of the same weight in the codomain: 

a : ^n{G/Uf. 

Note that = Gr^+(W^) = (Gr^+W)^. 
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There exists a unique G-module homomorphism a : Gr^+W 71{G/U) such that the 
following diagram commutes: 

a: ^ n{G/Uf 

n n 



a 



Gr^+W 



TZ{G/U) 



We wish to show that a is an algebra homomorphism, i.e., 



{Gi^^WY X (Gr^+W)' 



a. i 



a I 



rrtw 



(Gr^+W)^+'^ 



1,2, 



n{G/uY X n{G/uY m^^^^ n{G/uY+^' 

commutes. 

We have two maps: 

/, : (Gr^+W)^ ® (Gr^+W)^ ^ 7^(G/f/)^+^ 

defined by: (8) w) = rnTi{G /u){oi{y) ® a{w)) and /2(f ®w) = a{my^{v ®w)). 
Each of /i and /2 is G-equivariant and, 

dim {(3 \(3: {Gi^+Wf ® (Gr^+W)'' ^ 7^(G'/f/)^+^ } = 1 

because the Cartan product has multiplicity one in the tensor product of two irreducible 
G-modules V\ and 

Therefore, there exists a constant G such that /i = C/2. We know that a|wc^ = a is an 
algebra homomorphism. So for highest weight vectors G and w'^ G W^: 

(Note that v^w^ is a highest weight vector.) Note that G = 1. □ 



3.3. Dual Pairs and Duality Correspondence. We recall here some material and nota- 
tion that will be required for later sections. Much of this material can be found in several 
other sources on classical invariant theory (such as [H] and jTUj). 

In our context, the theory of dual pairs may be cast in a purely algebraic language. In 
this section, we will describe three dual pairs {K,q), where i^' is a classical linear algebraic 
group defined over C and g is a complex classical Lie algebra. In each case, we have a linear 
action of on a finite dimensional complex vector space V, which is a finite sum of copies 
of the standard module for K or copies of the dual of the standard module for K. This 
action induces an action on the complex valued polynomial functions on V, upon which 
acts by polynomial coefficient differential operators. The actions of g and K commute 
with each other. Furthermore, the algebra of polynomial coefficient differential operators 
which commute with the i^-action (resp. 0-action) on V{V) is generated as an algebra 
by the image of the g-action (resp. i^-action). In light of this situation, we may regard 
V{V) as a representation of q and K simultaneously. Theorem 3.4 describes, in part, the 
(multiplicity-free) decomposition of V{V) into irreducible modules for the joint action. 

Of particular importance are the if-invariants in V{V). In each case, we may describe this 
invariant ring through the action of g. Indeed, q may be decomposed into three subspaces 
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denoted Q^'^'^\ Q^^'^^ and g^^'^^ These subspaces are in fact Lie subalgebras with ^(O'^) and 
0^^'°) abehan, while q^^'^^ normahzes each of them. Furthermore, 0^^'^^©0*^^'°^ and g''^'^-' ©0*^°'^^ 
are the Levi decompositions of certain parabohc subalgebras of q. 

Theorem 3.3 asserts that V{V)^ is generated as an algebra by g*^^'°^ Moreover, within a 
certain stable range (described in Theorem 3.3), ViV)^ is isomorphic to iS(g*^^'°''), the full 
symmetric algebra on q^'^'^\ This is the first fundamental theorems of classical invariant 



Note that at the same time, we obtain an action of K (by conjugation) on the constant co- 
efficient differential operators on V{V), denoted 'D{y). In turn, the /^-invariant subalgebra, 
V{y)^ is generated by This brings us to our next ingredient. Define the K -harmonic 

polynomials to be: 



For each dual pair, we have a surjection, ViV)^ ® 7i ^ '^(^) defined by multiplication. 
Note that we may regard ViV) as a module over the algebra ViV)^ . By definition, this 
module is free iff m is injective. Within a certain stable range, m is indeed injective, and 
this range is indicated as part of Theorem 3.3. We have provided a proof of the injectivity 
part of this theorem (also known as the Separation of Variables Theorem) as an appendix 
of this paper. 

For each (g, K), the subspace g'-^'^-' is isomorphic to the Lie algebra of a subgroup G^^'^^ C 
GL{V) which commutes with the action of K. In Theorem 3.5 we describe the action of this 
group. Note that, in general, the differential of the action of G^^'^^ on ViV) is not quite the 
same as the action of g^^'^^; however, it differs only by a central shift. 

Under the joint action of x G^^'^\ 7i is a multiplicity- free invariant subspace of ViV). 
The precise decomposition of Ti. is provided in Theorem 3.5. 

Finally, in each of the three dual pair settings we have V{V) = I{J'^) ® H, where I{J^) 
is the ideal in V{V) generated by g(2'°) (which is the same as the ideal generated by the 
homogeneous invariants of positive degree). We note that the natural map: 



is a linear isomorphism of representations. 

Details in this section including all theorems stated can be found in ^Uj, ^2] or [T^ . 

3.3. L Definitions of the Three Dual Pair Actions. We now describe the three dual pairs in 
detail as well as state Theorems 3.3, 3.4 and 3.5 on a case-by-case basis. For the following, 
we let Mn,m be the complex vector space of n by m matrices. We shall select a coordinate 
system {xij\i = 1, ■ ■ ■ , n, j = 1, ■ ■ ■ , m}. 

CASE A: (0„,SP2^) where V := M„„,. 

By On we mean the group of invertible n x n matrices, g such that gJg^ = J where J is the 
n X n matrix: 



theory (see HU], d, 



n:={f e V{V) I A/ = for all A G V{V)^] . 



n 



V{V)/I{J^) 



(3.1) 







1 \ 



J 



1 







1 



V 1 



0/ 
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This group acts on the complex n X m matrices, V = Mn^m by left multiplication. 

Using the standard matrices entries as coordinates, we define the following differential 
operators: 

V" ^2 V" T T . ■ flTlH W — V T- Q 

We define three spaces: 

= Span {Eij + I Sij I i, j = 1, . . . , m} , 

= Span |r^j | 1 < « < i < iri^ , and 

= Span {Ay \ 1 < i < j < m} . 

The direct sum, q := sp^J^^ ®^p2m^ ®^p2m^ ^ preserved under the usual operator bracket 
and is isomorphic, as a Lie algebra, to the rank m complex symplectic Lie algebra, Sp2jn- This 
presentation defines an action of 5p2m 'P{Mn,m) ■ The 0„ action is defined by multiplication 
on the left: for g e On and / e V{Mn,m) we set g ■ f{x) — f{g*x) for all x e Mn,m- 



^V2m 
^P2m 



CASE B: (Sp2n,S02m) where V = Man,™- 

By Sp2n we mean the group of complex 2n x 2n invertible matrices, g, such that gJg^ = J 

where J — ^ ^ ^0 ^ with 1^ the k x k identity matrix. This group acts on the complex 

2n X m matrices, V = M2n,m by left multiplication. 

Using the standard matrix entries as coordinates, we define the following differential op- 
erators: 



D. 



52 



52 



dxs 



C2 

ij 



Es=l ('^**'^s+">J -^s+nji-^sj) ) arid 



E - - X 



We define three spaces: 



so 



so 



so 



(1,1) 

2m 

(2,0) 
2m 

(0,2) 
2m 



= Span {Eij + n 5ij 
= Span {Sij \1 <i < j <m} , 
— Span {Dij \ 1 < i < j < m} . 



i,j = 1, ...,m}, 
and 



The direct sum, q := sOj^'^'* ' 



■'2m 



)S0 



(1,1) 

2m 



^'^fm \ isomorphic to S02m; the rank m orthogonal 
Lie algebra of type D, and this presentation defines an action of S02m on V{M2n,m)- For 
g e Sp2„ and / e V{M2n,m), we set g ■ f{x) = f{g^x) for all x e M2„,m- 



CASE C: (GLn,0l„,+£) where V = M^^^n © M^,^. 

Let {xab} and {Hcd} be the coordinates on Mn^^ and M^yj respectively. Define the following 
differential operators: 

n q2 " "9 "9 
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(In the above, i and j range over the appropriate interval defined by the sizes of the matrices.) 
We define three spaces: 

flfi;;) := Span {Efj + f S^j | i, j = 1, . . . , m} © Span {Ef. + f S^j \ i, j ^ 1, ...,£} , 

sS? — Span {ffj I i = 1, . . .,m,j = !,...,£}, and 

glJUj Span {Aij | i = 1, . . .,m,j ^ !,...,£}. 

The direct sum, q := Ql^^'f ©sC'l'' ® S^mf ■> isomorphic to the rank m + i general linear 
Lie algebra Qlm+e, and this presentation defines an action of Qlm+e on V{Mn,m © ^£,n)- For 
g e GLn and / e P(M„,^ M^,„), we set ^ • f{x,y) = fig^x,y{g^)-^) for all x e M„,^, 

3.3.2. Theorems on the Invariants, Decompositions and Harmonics. Let SMm and AM;r„ be 
the space of symmetric and anti-symmetric m by m matrices respectively. If y is a vector 
space, we denote the symmetric algebra on V by S{V). Note that in each of the dual pairs, 
we have defined the action of K on V{V) so that ViV) = S{V) as K modules. (This is in 
contrast with the usual identification V{V) = S{V*).) Also, for a set S, we shall denote by 
C[S] by the algebra generated by elements in the set S. 

Theorem 3.3. (First Fundamental Theorem of Invariant Theory and Separation of Vari- 
ables) 

(a) CASE A: (0„,5p2m) The invariants 

Jr,,m := V{M^,mT- = CfrJ] 5(5P^5?^) ^ V{SM^) ifn>m). 

Let Tln,m Q 'P{Mn,m) dcnotc the On-harmonics. Further, if n> 2m, we have separa- 
tion of variables 

(b) CASE B: (Sp2n,-S02m) The invariants 

J2n,m := ViM2n,mf'"" = C[5j] 3(50^^^) ^ V{AMm) if U > m) . 

Let 7i2n,m ^ 'P{M2n,m) dcnotc the Sp2n-harmonics. Further, if n > m, we have 
separation of variables 

V{M2n,m) - 'H2n,m J2n,m- 

(c) CASE C: (GLn, glm+e) The invariants 

Jn,m,i := V{Mn,rr, © Me^nf"" = C[rl] (= S{gQ^) ^ V{M„,,i) tfn> min(m,£)) . 

Let 7in,m/ ^ 'P{.Mn,m © ^^,n) dcnotc thc GL„- harmonics. Further, if n > m -\- £, we 
have separation of variables 

We refer our readers to the Appendix for a new proof of the Separation of Variables 
Theorem. 

The Lie algebra q acts on V{V) via differential operators. Under this action V{V) decom- 
poses into irreducible (infinite-dimensional, highest weight) representations of g. The group 
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K is reductive, so V{V) also decomposes into irreducible (finite-dimensional, highest weight) 

representations of K. 

In the space of polynomial coefficient differential operators, the algebra generated by the 
image of the X-action is a the full centrahzer of the algebra generated by the g-action. A 
consequence of this is that V{V) has a multiplicity free decomposition under the joint action 
of the group K and the Lie algebra g. The irreducible constituents are of the form U <S> U 
where U is an irreducible /^-representation while U is an irreducible g-representation. 

In each of the three cases. On, Sp2n and GLn, denote the representations paired with E"^, 
and by E'^, and respectively. The parametrization involving highest weights 
being made precise by the pairing defined in the following theorem: 

Theorem 3.4. (Multiplicity-Free Decomposition under K. x q) 

For each case, we state the decomposition ofViV) into irreducible representations: 

(a) CASE A: (0„,5p0 

P(M„,„) = ® ^f2„o (3.2) 

where the sum is over all partitions A with length at most min (n, m), and such that 
(A')i + (A')2 < n. 
As a representation ofGLm, 

E{2m) = Jn,m " for any n,m>0, 

^ S{SMm) ® provided n > 2m. (3.3) 

(b) CASE B: (Sp2„,502m) 

Wn,J=0^(^„)®V(L) (3.4) 

where the sum is over all partitions A with length at most min {n, m) . 
As a representation ofGL^, 

V(2m) = J 2n,m ' FK for any n,m>0, 



^ S{AMm) F(K provided n>m. (3.5) 



(2m) — <J 2n,m ' ^ (m) 

?■ 
(m) 

(c) CASE C: (GL„,0l™+,) 

.(A+,A-) ^ ?;(A+,A-) 



r{Mn,m © M,,„) = fI^^ '''^ ® fI'^J-^ (3.6) 

where the sum is over all ordered pairs of partitions (A"*", A^) such that 
£(A+) + £(A-) < n, £(A+) < min (n, m), and £(A-) < min (n, t). 
As a representation of GL^ x GL(^, 

= Jn,m,e ■ (<) ® for any n,m,i> 0, 

^ S{Mm,e) ® ® F(^~ ) provided n>m + e. (3.7) 

Remcirks: In Case C, the representation Fj-^^^ ^ are (in general) complexifications of 
infinite-dimensional highest weight representations of {u(m,£))c — B^m+e- Sometimes want 
to emphasize the interplay of the two pieces M„„j and M^yj, by writing Qlm,t instead of 

15 ' ' 



gim+e- The degenerate case when £ — is particularly interesting. 
This is the GLn x GLm duality: 

V{M^,m) = i^('„) ® (3.8) 

A 

where the sum is over all integer partitions A such that i{X) < min (n, m). 

Theorem 3.5. (Multiplicity-Free Decomposition of Harmonics under K x G'-"'"'"'"-'^ 

We proceed in cases: 

(a) CASE A: (On,sp2m) -^^^ 'Hn,m ^ 'P{Mn^m) denotc the On-harmonics. The 
group On X GL^ acts on V{Mn,m) by {g,h) ■ f{x) = f{g^xh), where g e On, h e 
GLni and x e Mn^n^. Then 7in,m is invariant under this action. As an On x GL^ 
representation, 

V{Mn,m)/I{J^,J - nn,m = i^f.) ® (3.9) 

where the sum is over all partitions A with length at most min (n, m) and such that 
(A')i + (A')2 <n. _ 

(b) CASE B: (Sp2n,S02m) Let 7i2n,m 'P{M2n,m) deuote the Sp2n-harmonics. 
The group Sp2n x GL^ acts on V{M2n,m) by {g,h) ■ f{x) = f{g^xh), where g e 
Sp2n, h G GLni and x e M2n,m- Then 'H2n,m is invariant under this action. As a 
Sp2n X GLni representation, 

r{M2n,m)/l(jt,J = n2n,m = ® (3.10) 

where the sum is over all partitions A with length at most min (n, m) . 

(c) CASE C: (GLn,0U+^) Let Hn,m/ C V{Mn,m © Me^n) denote the 
GLn-harmonics. The group GLn x GL^ x GL^ acts on V{Mn,m © -^£,n) by 

{g, hi, h2) ■ f{x, y) = fig'xhi, h\y{gy^), 

for g e GLn, hi G GLn^, /i2 G GLi, x G Mn,m andy G M^^n- Then7in,m,i is invariant 
under this action. As a GLn x GL^ x GL^ representation, 

V{Mn,m © M,,J//( J„+^_,) - nn,m,i = ^^''"^ (<) ® ) (^-H) 

where the sum is over all ordered pairs of partitions (A^, A^) such that 
i{X+)+i{\-) < n, f:{X+) < min (n,m), and < min(n,£). 

Remarks. The three cases are summarized in the following table: 



K 


On 


Sp2n 


GLn 





5p2m 


502m 




V 




M2n,m 


Mn,rn © M^,^ 




GLm 


GLjn 


GLm X GLi 
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4. Reciprocity Algebras 



In this paper, we study branching algebras using classical invariant theory. The formula- 
tion of classical invariant theory in terms of dual pairs ^21 allows one to realize branching 
algebras for classical symmetric pairs as concrete algebras of polynomials on vector spaces. 
Furthermore, when realized in this way, the branching algebras have a double interpretation 
in which they solve two related branching problems simultaneously. Classical invariant the- 
ory also provides a flexible means which allows an inductive approach to the computation 
of branching algebras, and makes evident natural connections between different branching 
algebras. 

The easiest illustration of the above assertions is the realization of the tensor product 
algebra for GLn presented as follows. 



4.1. Illustration: Tensor Product Algebra for GL„. This first example is in [14] . which 
we recall here as it is a model for the other (more involved) constructions of branching 
algebras as total subalgebras of GLn tensor product algebras. 

Consider the joint action of GL^ x GL^ on the 'P(M„ m) by the rule 

{g, h) ■ f{x) = fig^xh), for g e GL^, h e GLm, x G M„,„. 

For the corresponding action on polynomials, one has the decomposition (see Theorem 3.4(c) 
and (3.8)) 

P(M„,J^0F(t)®F(t,), (4.1) 

A 

of the polynomials into irreducible GLn x GLm representations. Note that the sum is over 
non- negative partitions A with length at most min {n,m). 

Let Um = UoLm denote the upper triangular unipotent subgroup of GLm- From decom- 
position (4.1), we can easily see that 

Since the spaces (-^(m))^"" one-dimensional, the sum in equation (4.2) consists of one 
copy of each F^^y Just as in the discussion of §3.2, the algebra is graded by ^4+, where Am 
is the diagonal torus of GLm, and one sees from (4.2) that the graded components are the 

By the arguments in §3.2, P(M„ „i)^'" can thus be associated to a graded subalgebra in 
7l{GLn/Un), in particular, this is a (0, l)-subalgebra as in Definition 3.1. To study tensor 
products of representations of GLn, we can take the direct sum of Mn^m and Mn/. We 
then have an action of GLn x GLm x GLi on V{Mn,m © Mn,e). Since V{Mn,m © Mn,e) ^ 
V{Mn,m) © PiMn/), wc may deduce from (4.1) that 

V{Mn,m © M„,,)^-><^^ ~ V{Mn,mf"' © V{Mn,if' 

Fin)) © ({Fli^f- © iFi:,)f') . (4.3) 
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Thus, this algebra is the sum of one copy of each tensor products F^^y Hence, if 
we take the f/„- invariants, we will get a subalgebra of the tensor product algebra for GLn- 
This results in the algebra 

This shows that we can realize the tensor product algebra for or more precisely, 

various (0, l)-subalgebras of it, as algebras of polynomial functions on matrices, specifically 
as the algebras V{Mn,m © Mn,^)^-^^^^^". 

However, the algebra V{Mn^m © Mn/)^"^^^^^^" has a second interpretation, as a different 
branching algebra. We note that M„_m©M„_^ ~ M„^m+^. On this space we have the action of 
GLn X GLm+e, which is described by the obvious adaptation of equation (4.1). The action of 
GLn X GLm X GLi arises by restriction of the action of GLm+e to the subgroup GLm x GLi 
embedded block diagonally in GLm+e- By (the obvious analog of) decomposition (4.2), we 
see that 

V{Mn,m+ef- ^ ©(i^H)""" ® 

A 

This algebra embeds as a subalgebra of 7l{GLm+e/Um+e), in particular, this is a (0,1)- 
subalgebra as in Definition 3.1. If we then take the Um x Ue invariants, we find that 

(P(M„,^+,)^")^-x^^ ^ 0(^h)^" © iFUe)f 

A 

is (a (0, l)-subalgebra of) the (GLm+e, GLm x GLi) branching algebra. Thus, we have es- 
tablished the following result. 

Theorem 4.1. (a) The algebra V{Mn,m+e)^"^'^"^^^'' is isomorphic to a (0,1) -subalgebra 
of the {GLn X GLn, GLn) branching algebra (a.k.a. the GLn tensor product algebra), 
and to a (0, l)-subalgebra of the {GLm+e, GLm x GL() branching algebra. 
(b) In particular, the dimension of the ip^ x ipf^ x ip^ homogeneous component for An x 
Am X Ai of V{Mn,m+e)^''^^"'^^'^ rccords simultaneously 

(i) the multiplicity of F^^-^ in the tensor product F^^^ © Fp^^-^ , and 

(ii) the multiplicity of Fl^^ ® ^(e) ^{m+e)' 

for partitions fi, v, A such that i{fi) < min(n,m), £(z/) < min(?7,,£) and i{X) < 
min(n, m + i). 

Thus, we can not only realize the GLn tensor product algebra concretely as an algebra 
of polynomials, we find that it appears simultaneously in two guises, the second being as 
the branching algebra for the pair {GLm+e, GLm x GL^). We emphasize two features of this 
situation. 

First, the pair {GLm+e, GLm x GLe), as well as the pair {GLn x GLn, GLn), is a symmetric 
pair. Hence, both the interpretations of V{Mn,m+e)^''^^"^^^^ are as branching algebras for 
symmetric pairs. 

Second, the relationship between the two situations is captured by the notion of "see-saw 
pair" of dual pairs jH]. Precisely, a context for understanding the decomposition law (4.1) 
is provided by observing that GLn and GLm (or more correctly, slight modifications of their 
Lie algebras) are mutual centralizers inside the Lie algebra sp(M„ „i) (of the metaplectic 
group) of polynomial coefficient differential operators of total degree two on M„ m [T2] [T^ . 
We say that they define a dual pair inside sp(M„ ,„)• The decomposition (4.1) then appears 
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as the correspondence of representations associated to this dual pair [T2] . Further, the pairs 
of groups {GLn,GLm+e) = {Gi,G[) and {GLn x GLn,GLm x GLe) = ((^2,(^2) both define 
dual pairs inside the Lie algebra sp{Mn^m+e)- We evidently have the relations 

Gi = GLn C GLn X GLn = G2, (4.4) 

and (hence) 

G[ = GLm+i D GLm X GLi = G^- (4-5) 
We refer to a pair of dual pairs related as in inclusions (4.4) and (4.5), a see-saw pair of dual 
pairs. 

In these terms, we may think of the symmetric pairs (G2,G'i) and {G'i,G'2) as a ^^recip- 
rocal pair'^ of symmetric pairs. If we do so, we see that the algebra V{Mn,m+i)^"^^'^^^'^ is 
describable as V{Mn^m+i) - it has a description in terms of the see-saw pair, and in 

this description the two pairs of the see-saw, or alternatively, the two reciprocal symmetric 
pairs, enter equivalently into the description of the algebra that describes the branching law 
for both symmetric pairs. For this reason, we also call this algebra, which describes the 
branching law for both symmetric pairs, the reciprocity algebra of the pair of pairs. 

It turns out that any branching algebra associated to a classical symmetric pair, that is, a 
pair (G, H) in which G is a product of classical groups, has an interpretation as a reciprocity 
algebra - an algebra that describes a branching law for two reciprocal symmetric pairs 
simultaneously. Sometimes, however, one of the branching laws involves infinite-dimensional 
representations. 

4.2. Symmetric Pairs and Reciprocity Pairs. In the context of dual pairs, we would like 
to understand the (G, H) branching of irreducible representations of G to if, for symmetric 
pairs (G, H). Table I lists the symmetric pairs which we will cover in this paper. 

If G is a classical group over C, then G can be embedded as one member of a dual pair in 
the symplectic group as described in jT^j. The resulting pairs of groups are {GLn, GLm) or 
{On, Sp2m), each inside Sp2nm, and are called irreducible dual pairs. In general, a dual pair 
of reductive groups in Sp2r is a product of such pairs. 
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Table I: Classical Symmetric Pairs 



Description 


G 


H 


Diagonal 


GLn X GLn 


GLn 


Diagonal 


On X On 


On 


Diagonal 


Sp2n X Sp2n 


Sp2n 


Direct Sum 


GLn+m 


GLn X GL„i 


Direct Sum 


On+m 


On X Oni 


Direct Sum 


Sp2{n+m) 


Sp2n X Sp2ni 


Polarization 


02n 


GLn 


Polarization 


Sp2n 


GLn 


Bilinear Form 


GLn 


On 


Bilinear Form 


GL2n 


Sp2n 



Proposition 4.2 Let G be a classical group, or a product of two copies of a classical group. 
Let G belong to a dual pair {G, G') in a symplectic group Sp2m- Let H C G be a symmetric 
subgroup, and let H' be the centralizer of H in Sp2m- Then {H,H') is also a dual pair in 
Sp2m, o-nd G' is a symmetric subgroup inside H' . 

Proof: This can be shown by fairly easy case-by-case checking. The basic reason that 
{H,H') form a dual pair is that, for any classical symmetric pair {G,H), the restriction of 
the standard module of G, or its dual, to is a sum of standard modules of H, or their 
duals P2]- This is very easy to check on a case- by-case basis. The see-saw relationship of 
symmetric pairs organizes the 10 series of symmetric pairs as given in Table I into five pairs 
of pairs. These are shown in Table II. □ 

Table II: Reciprocity Pairs 



Symmetric Pair (G, H) 


(H, fl') 


(G,b') 


{GLn X GLn, GLn) 


{GLn,Qlm+l) 


{GLn X GLn,Qim®QU) 


{On X On, On) 


(0„,5p2(m+^)) 


{On X 0„,Sp2m e5p2i) 


{Sp2n X Sp2n, Sp2n) 


(5p2n,S02(m+f)) 


{Sp2n X Sp2n,502m ©502^) 


{GLn+m, GLn X GLm) 


{GLn X GLm, Qk®Qk) 


{GLn+m, Q^i) 


{On+m, On X Om) 


{On X Om,5p2^ esp2^) 


{On+m, Sp2£) 


iSp2(n+m),Sp2n X Sp2m) 


{Sp2n X 5'p2m,S02£ ©S02^) 


{Sp2(n+m)-,^02e) 


{02n, GLn) 


{GLn, 0(m,m) 


(02n,Sp2m) 


{Sp2n,GLn) 


{GLn, Q^m,m) 


{Sp2n,502m) 


{GLn, On) 


(On,Sp2m) 


{GLn,Qlm) 


{GL2n,Sp2n) 


{Sp2n,502m) 


{GL2n,8im) 



Remark: Note that when the second component of any pair in Table 11 is of Lie type A, 
then the action actually integrates to the group. Table II also amounts to another point of 
view on the structure on which ^H] is based. 

We begin with discussions of reciprocity algebras in the next three sections. The discus- 
sions provided are ordered more in terms of complexity and do not follow the sequence given 
in Table I. 
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5. Branching from GL„ to O, 



Consider the problem of restricting irreducible representations of GLn to the orthogonal 

group On- We consider the symmetric see-saw pair {GLn, On) and {Sp2m,GLm). As in 
the discussion of §4.1, we can realize (a (0, l)-subalgcbra of) the coordinate ring of the flag 
manifold GLn/Un as the algebra of L^^-invariants on V{Mn,m)- If then look at the Uo„- 
invariants in this algebra, then we will have (a certain (0, l)-subalgebra of) the {GLn, On) 
branching algebra. Thus, we are interested in the algebra 

We note that, in analogy with the situation of §4.1, this is the algebra of invariants for the 
unipotent subgroups of the smaller member of each symmetric pair. 

Let us investigate what this algebra appears to be if we first take invariants with respect 
to Uo ■ We have a decomposition of V{Mn m) as a joint 0„ x 5p2m-iiiodule (see Theorem 
3.4 (a)): 

nMn,m)^®E^^^^®El^^y (5.1) 

Recall that the sum runs through the set of all non-negative integer partitions // such that 
£(//) < min(n, m) and (/i')i + (//')2 < n. Here E'^'^^ denotes the irreducible 0„ representation 
parameterized by ji. Recall from §3.3, the decomposition V{Mn,m) — ©^-^(n) ® ^{m)- '^^^ 
module E^^-^ is generated by the GLn highest weight vector in F^^y Further, E^^^^ is an 
irreducible infinite-dimensional representation of 5p2m with lowest glm-type F^jn)- 

Theorem 5.1. Assume n > 2m. 

(a) The algebra V{Mn,m)^°'^^^"' is isomorphic to a (0,1) -subalgebra of the {GLn, On) 

branching algebra, and to a {{],!)- subalgebra of the {5p2rn,GL,n) branching algebra. 

(b) In particular, the dimension of the (f)^ x ■?/''^ homogeneous component for Aq^ x A^n 
of V{Mn,m)^°^^^'^ records simultaneously 

(i) the multiplicity of E'^^^ in the representation F^y and 

(ii) the multiplicity of F^-^ in E'^^^-^ . 

for partitions /i, A such that i{ii) < m, l{\) < m. 

Proof. Taking the [/o„ -invariants for the decomposition (5.1), we find that 

where the sum is over partitions ii such that £(/x) < min(n, m) and (/i')i+(/i')2 < n. Note that 
the stability condition n > 2m guarantees the latter inequahty. The space (£'|^p'^o» is the 
space of highest weight vectors for {E^^^)^Ori . We would hke to say that it is one-dimensional, 

so that V{Mn,m)^°'' would consist of one copy of each of the irreducible representations E'^2m)- 
But, owing to the disconnectedness of 0„, this is not quite true, and when it is true, the 
highest weight may not completely determine E^^y 

However, if n > 2m, then (£'|^p^°" is one-dimensional, and does single out among 
the representations which appear in the sum (5.1). Hence, let us make this restriction for 
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the present discussion. Taking the Um invariants in the sum (5.2), we find that 

{nM^,mro.fr. ^ 0(Ej^^)^o„ ^ (E^^^^f-. (5.3) 

Note that the sum is over all partitions fj, such that ^(//) < m (since n > 2m). The space 
(-^(2m))^'" describes how the representation E'^2m) -^Psm decomposes as a module, or 
equivalently, as a GL^- module. In other words, (-E'^m))^'" describes the branching rule from 
Sp2m to Qljn for the module -B^^). 

We know (thanks to our restriction to n > 2m) that the space (£'^^)^0" is one-dimensional. 
Let (f)'^ be the Ao„ weight of {E^^^^^)^'^". Thus, 0^ is the restriction to the diagonal maximal 
torus Ao„ of the character ip'^ of the group An of diagonal n x n matrices. Our assumption 
further implies that (f)'^ determines Therefore, for a given dominant A^ weight 

corresponding to the partition A, where i{X) < m, the ■?/''*'-eigenspace in {E^^m))^"' ^^^^^ 
the multiplicity of in the restriction of E^^^^ to glm- This is the same as the dimension 
of the joint {4"^ x ■?/''^)-eigenspace in 

(P(M„,^)^°")^'" -^(M,,^)^^"^^^™ ~ {V{Mn,mf-f''-- 

But we have already seen that this eigenspace describes the multiplicity of E^^^ in F^y 

Thus, again the Ao„ x A^ homogeneous components of 'P(M„ „j)'^°"^'^'" have a simultane- 
ous interpretation, one for a branching law associated to each of the two symmetric pairs 
composing the symmetric see-saw pair. □ 

In this case, one of the branching laws involves infinite-dimensional representations. How- 
ever, they are highest weight representations, which are the most tractable of infinite- 
dimensional representations, from an algebraic point of view. 

6. Tensor Product Algebra for 0„ 

Using the symmetric see-saw pair ((0„,0„ x On), {Sp2m x Sp2i) Sp2{m+e.)}) i we can con- 
struct ((0, l)-subalgebras of) the tensor product algebra for On- To prepare for this, we 
should explicate the decomposition (5.1) further. 

Let us recall the basic setup as in §3.3.1 Case A. Recall that Jn,m = '^(^n.m)*^" is the 
algebra of 0„-invariant polynomials. Theorem 3.3(a) implies that Jn,m is a quotient of 

5(sp2m'''')' the symmetric algebra on sp^2m ■ 
The natural mapping 

Hn,m - V{Mn,m)II{J:,J (6.1) 

is a linear 0„ x GL^-modulc isomorphism. Further, the 0„ x GLm structure of 'Hn,m is as 
follows (see Theorem 3.5(a)): 

nn,m^®El^®Fi;^y (6.2) 

Here fi ranges over the same diagrams as in (5.1). 
Prom Theorem 3.4(a), 

El2m) ^ Fi;m) ■ Jn,m ^ S{,pt^^) ■ F^^^^ , (6.3) 
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and it follows that _ _ 

In other words, we can detect the Sp2m isomorphism class of the module E'^2m) 

isomorphism class of the quotient E^^m)/^'^^^^^ ' -^(2m))- ^^^o, if C V{Mn,^) is any 
5p2m-iiivariant subspace, then 

and this subspace also reveals the 5p2m isomorphism type of W . 

We can use the above to find a model for (a (0, l)-subalgebra of) the tensor product 
algebra of On- One consequence of the above discussion is that 

consists of one copy of each irreducible representation E^^^ . 

If we repeat the above discussion for Mn/-, and combine the results, we find that 

(P(M„,„)/J(J+J)^- ® [V^Mn.Ml^JlSf' 

- [Kn) ® ^(n)) ® ((^m)''"' ® (^W)^^) (6-5) 

is a direct sum of one copy of each possible tensor product of an E^^-^ with an E^^^y At this 
point, we make the assumption that n > 2(m + £), as in this range the 0„ constituents of 
decomposition are irreducible when restricted to the connected component of the identity in 
On- If we now take the f/o„ -invariants in equation (6.5), we will have (a (0, l)-subalgebra 
of) the tensor product algebra of 0„: 

((P(M„,^)//(J-+J)^- ® {V{Mn,t)/I{J:i,dff"'' 

^ © (^fn) ® ^(n)) ® ((^(1))^- ® (^W)'^'') • (6-6) 

We can describe this algebra in another way. Begin with the observation that V{Mn,m) 'S> 
V{Mn,i) c^V{Mn,m+e), a.nd 

V{Mn,m)/I{J:,J P(M„,,)//( ) C V{Mn,m^t)/I{J^^n, ® ). 

Thus 

{V{Mn,m)/I{JtJf- ® {nMn,i)II{J:,,)f' ^ {r{Mn,m+e) / I{Jrl,m ® Jrt,e)f , 

and 

mMn,n.)/I{Jn^,m)r- ® ( ^4,) //(JC^) )^0 

^ ((^(M„,^+,)//(J-+^© Ji))^-><^0^°" 

^ {{V{Mn,m^e)/I{j:,n, © J^))^-") '"'"'''^ ■ 
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Theorem 6.1. Given positive integers n, m and i with n > 2{m + €) we have: 

(a) The algebra 

is isomorphic to a (0, l)-subalgebra of the {On x 0„, 0„) branching algebra (a.k.a. the 
On tensor product algebra), and to a {0,1) -subalgebra of the (5p2(m+£) ' ^P2m ® ^p2e) 
branching algebra. 

(b) Specifically, the dimension of the ((p^ x ip^^ x ip'^ )-eigenspace for Ao„ x x of 

{{V{Mn,„^+£) / I{Jn.m® Jn^i))^°''Y"'^^' Tccords simultancously 

(i) the multiplicity of E^^-^ in E^_^~^ E^n)> '^■^ ''-'-'^^^ '^■^ 

(ii) the multiplicity of E'^^^^ ® E^2t) restriction of E'^2{m+t)) ■ 
Here the partitions fi, u, A satisfy the following conditions: 

£{fi) < min(n, m), l{v) < min(r2, i), and i{X) < mm{n, m + €). 

Proof. Let us now compute the ring expressed in this way. From Theorem 3.4(a), we know 
that 

® E^2m) ^ 0(^fn))^°" ® E^2my 

Note that within the range n > 2{m+i) we have dim{E^^^)^'^" = 1 since the (^^-representations 
E^_^.^ remain irreducible when restricted to S On- 
Now repeat this with m replaced by m + i: 

V{Mn,m+,T°- ^ f ^fn) ® E>l2(m^i))\ ^ 0(^fn))''°" ® E^2(m^i)y 



Hence 

{r{Mn,mu)/I{J:,m © ^nt.))''°" ^ (^0 ^n) ® ^(2(m+£)) ^ I KJ^m © ^n' 

^ 0(^(n))^°" ® (^f2(m+.))/(^P?m°^ ©^P£'°^) • ^f2(m+.))) ■ 
\ 

Prom this we finally get 



UOn 



{{V{Mn,m+e)/I{Jrl,m © Jrt,i) f''"Y 

^ ©(^n))"""" ® (^f2(m+.))/(^P?^ ® 5P£'°^) • ^f2(m+.))) 
A 

Prom the discussion following equation (6.1), we see that the factor 

(e^ /r^t)(''°) ffi ati^''^^ . E^ 

[^{2{m+e))/ {^P2m ^P2e ) ^{2{m+e)) J 

tells us the Sp2m © Sp2£ decomposition of -E'(^2(m+^)) ■ '-' 
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Hence, again the algebra has a double interpretation, one in terms of decomposing tensor 
products of On representations, and one in terms of branching from 5p2(m+t) to Sp2m ® Sp2i 
(although the second branching law involves infinite-dimensional representations). 

7. More Reciprocity Algebras for GLm) 

Whereas our first example of a reciprocity algebra in §4.1 involved only finite-dimensional 
representations, the others all involve infinite-dimensional representations in some respect. 
It turns out that the apparently exceptional nature of the reciprocity algebra for the pair 
{GLm GLm) is somewhat deceptive. In fact, we can associate several reciprocity algebras to 
{GLm GLm), and nearly all of them will involve infinite-dimensional representations. 

We shall refer to §3.3.1 and consider the action of GLn on V{Mn,m ® -^£,n) by the rule 

9-f{x,y)^f{9'x,y{g')-') (7.1) 

for X e Mn^m, y £ Mi^n ^ud g e GLn- Recall from Theorem 3.3(c) that the algebra Jn,m,i 
generated by It is the space of all polynomials on P(M„^j„©M^^„) invariant under GL^. 

Let TCn,m,e be the space of GL^-harmonics and recall the GL„ x GL^ x GL^ isomorphism 
(see Theorem 3.5(c)): 

V{M^,m ® Me,^)/I{J+^^,) ~ nn,m,e. 
Theorem 7.1. (a) The algebra 

{{V{Mn,m © M,,„)//( J„+„^,))^-x^^ ® {r{Mn,m' © M,,n)/I{J;l,mV')f-'"''"y^ 

is isomorphic to a {0,1) -subalgebra of the {GLn x GLn,GLn) branching algebra as 
well as to a {Qlm+m' /+£' , 9^m/ ©0^m',£') branching algebra. 
(b) In particular, the dimension of the (An x Am x Am' x x A^i)-eigenspace of 
V{Mn,m+m' ® ^i+i' ,n) dcscribcs simultaneously 

(i) the multiplicity ofF^^^'^~^ in F^^^^'"'^ (g) F^^^'''~\ and 

(ii) the multiplicity of the representation f'^'^^^ ■* ® f'^^,'^,^ ■* of glm,e © 0^^',^ the 

restriction of the representation of Ql(m+m'),{e+e')- 

Here the partitions ix^ , , v'^ , v~ , \^ , \~ are such that i{iJ>'^) < m, i{iJ>~) < i, 

i{fi+) +i{fi-) < n, £{u+) < m', e{u-) < £(i/+) + £(i/-) < n, e{X+) < m + m' , 
i{X ) <£ + £' and £{X+) + £{X-) < n. 

Reiiicirks: Recall from the remarks after Theorem 3.4 that we have written glm,e instead of 
Qlm+i to emphasize the interplay of the two components M^^m a-^d M^ „. 

Proof. Prom the above description of V{Mn,m © Mi,n), we can see using (3.11) that 

{V{Mn,m © M,,0//(J„:^,,))^'"^^^ ^ <f ® (F(';"/™ ® {Fi;^)^^ 

is a multiplicity-free sum of representations f'^^^'^ ^ of GLn- Again, this can be embedded 
as a (0, l)-subalgebra of the coordinate ring of GLn/Un- 

Now repeat this with m' in place of m and £' in place of £. We again get a multiplicity-free 
sum of a family of representations of GL„. If we take the tensor product of the two sums, 
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and look at highest weight vectors for GLn, we will get a (0, l)-subalgebra of the tensor 
algebra for GL„: 

where the sum is over partitions fi~^, fi~ , and v~ such that ■^(z^''') < ^{iJr) < £, 
+ /(//-) < n, £(z/+) < m', < f , + < n. 

On the other hand, 



U„xUm,y<UexU^,xU(, 

{r{Mn,m © M,,„)//( j;^,,)) ® (P(M„,„, © M,, „)//( J-+^, „)) 



- © M,+,, „)//( © 



'/ri xUmXU(X U^i X U^i 

UmXUexU^/xU^/ 



, A+,A- 



(A+,A-) \ C^mXC/<x[/^,x(7<,, 



A+,A- 

which tells us about the ^\m,t © ^^m',i' decomposition in the representation of 
0^(m+m'),(^+^')- This completes the proof. □ 

The construction of §4.1 of course is just the case ^ = = of the current discussion. 
That case is notable for staying completely in the context of finite-dimensional representation 
theory. Another case of interest is when £ = = m' . Then, although the representations of 
^imi' ai'c infinite dimensional, the representations of the subalgebras gt^ and gl^/ are finite 
dimensional. This case is analogous to branching from GL„ to On (or from GL2n to Sp2n)- 



8. The Stable Range and Relations Between Reciprocity Algebras 

Let us summarize our discussions this far. Given any classical symmetric pair, we can 
embed it in a (family of) see-saw symmetric pair(s). Doing this, we find that (a (0,1)- 
subalgebra of) the branching algebra for the pair can equally well be interpreted as the 
branching algebra for a dual family of representations of the dual symmetric pair. The 
representations of the dual symmetric pair will frequently be infinite dimensional, but they 
are always highest weight modules. 

An immediate consequence of this isomorphism of algebras is the isomorphisms of inter- 
twining spaces and hence equality of multiplicities, which we have collectively described as 
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reciprocity laws. These reciprocity laws are of the same nature as Frobenius Reciprocity for 
induced representations of groups. 

From §4.2, we see that the see-saw symmetric pairs actually come in two parameter fam- 
ilies. If one of the pairs involves many more variables than the other, then certain features 
of the discussions above become simpler. 

Take the results of Theorem 4.1 as an illustration: The Littlewood- Richardson coefficients 
for GLn, 

c^, = dimHomGL„(F(t), i^fn) ® i^^) 

= dimHomcL^xGL.li^f^) ® F^,^, 

are independent of n, if n > m + and depend only on the shape of the partitions /x, u and 
A. 

Consider another example: branching from GL2n to Sp2n- If we let these groups act on 
'P{M2n,m), we get the see-saw pairs {Sp2n,S02m) and {GL2n, Qim)- The branching coefficients 
from GL2n to Sp2n can be described as follows: 

F{2n) \sp2n= X^'^A ^(2n) 
A* 

where 

dx = dimHom5p2„(V'2n>^2n) 

= dimHomcL^ {Ftm),F(i^ ® Sisot'^)) 

= dimHomcL^ ® 5(A^C'")) 

is independent of n, if n > m, and only depends on the diagrams A and fi. This allows one 
to create a theory of "stable characters" for Sp2n- Similar considerations apply to GL^ and 
On and this idea has been actively pursued by jlZj, amongst others. 

These are all instances of stability laws. The well-known one-step branching from GLn 
to GLn^i is another instance. This branching can be described entirely by diagrams, with 
no mention of the size n, if n is large. Iteration of this branching also shows that when n 
is large, the weight multiplicities of dominant weights of an irreducible GLn representation 
are independent of n. See jT] for the other classical groups, which don't share this stability 
property. 

In the sections that follow, we will illustrate the simplifications that occur in the stable 
range, highlighting certain specific see-saw pairs. In all these cases, we show that the branch- 
ing algebras associated to symmetric pairs can all be described by use of suitable branching 
algebras associated to the general linear groups. Thus, if we can have control of the solution 
in the general linear group case, we will have some control of the other classical groups. The 
other non-trivial examples will be important extensions of this work, and we hope to see 
them in further papers, for example, JHIj and JBjI- 

9. Stability for Branching from GL„ to 0„ 

We begin with a detailed discussion of the case of {GLn, On) and (sp2m,G'Lm). Here we 
have already encountered the stable range, without the name. It is when n > 2m. Several 
things happen in the stable range: 

27 



(a) The representations E^^-^ of the orthogonal group remain irreducible when restricted 
to the special orthogonal group SOn, and furthermore, no two of them are equivalent. 

(b) Recall the algebra J^n,m of 0„-invariant polynomials on Mn,m generated by the qua- 
dratic invariants, which is the abelian subalgebra sp^2m^ of sp2m- In the stable range 
(in fact it holds true whenever n > m), the natural surjective homomorphism 

is an isomorphism. See Theorem 3.3(a). 

(c) In the stable range, the multiplication map 

® Jn,m — 'Hn,m ® «S(sp2^°^) ^ V{Mn,m) 

is also an isomorphism of 0„ x GLm-modules. See Theorem 3.3(a). 

Of course, the subspace Ti.n,m of harmonic polynomials is not an algebra - it is not closed 
under multiplication. This is quite clear, since Hn,m contains all the linear functions, which 
generate the whole polynomial ring. However, to form the reciprocity algebra associated to 
the symmetric see-saw pairs {GLn, On) and {sp2m, GLm), we need to take the C/o„-invariants. 
Thus, our reciprocity algebra is a subalgebra of 

^ (|0(^(n))^°" ® ® S{,pf^\ (9.1) 

Theorem 9.1. When n > 2m, the space Hn^ is a subalgebra of V{Mn,m)- Hence, the 
algebra V{Mn^jn)^°n is isomorphic to a tensor product 

V{Mr,,mf''-^nli^®S{spt'^) 

of the algebras and S{sp'^^^). Furthermore, the algebra is isomorphic (as a 

representation) to the subalgebra TV~{GLjn/Um) of TZ{GL^/Um) defined by the polynomial 
representations. 

Proof. Note that 7^^*^ can be identified with a subalgebra TV'{GLm/Um) of TZ{GL^/Um) 
defined by the polynomial representations, from our discussion in §3.2. Consider the space 
of polynomials belonging to the sum in the last expression of equation (9.1). Let 
{xjk I j = I, . . . ,n,k = 1, . . . ,m} be the standard matrix entries on Mn^rn- In order to 
make the unipotent group Uq^ of On maximally compatible with (in fact, contained in) the 
unipotent subgroup f/„ of GLn, we choose the inner product on as in Section 3.3.1. By this 
choice, joint 0„ x GL^ harmonic highest weight vectors are monomials in the determinants 



5j = det 



Xii Xi2 ... Xij 

X21 X22 ■ ■ ■ Xij 



Xji Xj2 ■ ■ ■ Xjj_ 
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for j = 1, . . . , m. (9.2) 




as {&i,&2,&3, • • • ,bj} ranges over all j-tuples of integers from 1 to m. Indeed, the span of 
such monomials is clearly invariant under gl^, and consists of highest weight vectors for 0„. 
Finally, we see that these monomials will all be harmonic, because the partial Laplacians 
spanning Sp2m^^ have the form 



Since every term of A^^ involves differentiating with respect to a variable Xjk with j > n/2, 
and the determinants (9.3) do not depend on these variables, we see that they will be 
annihilated by the Aa^, which means that they are harmonic. This shows that Tin^ is a 
subalgebra of V{Mn^m)- 

We have thus completed the proof of the theorem. □ 

We can use the description in Theorem 9.1 of V{Mn,m)^°" to relate the branching algebra 
V{Mn,m)^°"^^'^ to the tensor product algebra for GLm- As a GL^-module, the space 
sp^2m^ is isomorphic to S^{C"^), the space of symmetric m x m matrices. It is well known 
that the symmetric algebra iS(iS^(C™')) is multiplicity-free as a representation of GLm, and 
decomposes into a sum of one copy of each polynomial representation corresponding to a 
diagram with rows of even length (or a partition of even parts) : 



(Note that this result is in several places in the literature. See ^U] and ^3] for example.) 

As a GLm,-module, iS(iS^(C™')) could be embedded in 7l{GLm/Um), but the algebra struc- 
tures on these two algebras are quite different. 

Using the dominance filtration (see §3.2), we have a canonical y4+-algebra filtration on 
iS(iS^(C'")). If we form the associated graded algebra, then Theorem 3.2 says that it will 
be isomorphic to the subalgebra of R{GLm/Um) spanned by the representations attached to 
diagrams with even length rows. 

Let us denote the associated graded algebra of 5(5^ (C")) by Gr^+5(iS^(C'")). Let us 
denote the subalgebra of lZ{GLm/Um) spanned by the representations attached to diagrams 
with even length rows by 7l^'^{GLm/Um)- 

We can filter the tensor product ?in,m ®'5(sp2m°^) ^y means of the filtration on iS(sp2^°''). 
The associated graded algebra will then be Hn'^ ® Gr^+5(sp2^°^). This discussion has 
indicated that the following result holds. 

Theorem 9.2. When n > 2m, the associated graded algebra of V{Mn^rn)^°" with respect to 
the dominance filtration on the factor Jn,m is isomorphic to the tensor product of the graded 
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(9.4) 



5(5^(C™))^0F( 



(9.5) 



V 



subalgebras 7l'^{GLm/Um) and 7l'^'^{GLm/Um) of 7l{GLm/Um) : 

G'r^+(P(M„,„)^°'0 ^ n+{GLjUm)®n+HGLjUm). 

Of course, Gr^+ (P(M„ is isomorphic as a GLm-niodule to P(M„„i)^°" in an 

obvious way, by construction. Also Gi^+ {V{Mn,m)^°'') inherits the Aq^ grading from 
V{Mn,m)^°" ^ it becomes identified with the ^4+ grading on the first factor {GL^n/Um) in 
the tensor product of Theorem 9.2. On the other hand, the second factor is also A^-graded 
in the obvious way, since it is the factor which defines the associated graded. When we 
take the Um invariants, we get another grading by associated to the Am action on the 
Um invariants. This triply yl+-graded algebra is evidently a (0, l)-subalgebra of the tensor 
product algebra of GLm- 

On the other hand, we could take the Um invariants inside 'P(M„^m)^°", and then pass to 
the associated graded. It is not hard to convince oneself that these two processes commute 
with each other. Hence, we finally have: 

Corollary 9.3 When n > 2m, the associated graded algebra ofUm invariants in V{Mn,m)^°'^ , 

- (n+iGLm/Um) ^n^\GLm/Um)f"' 

is a triply-graded (0, l)-subalgebra of the tensor product algebra of GLm- The restrictions on 
the gradings which define Gr^+ (^(V {Mn^m)'^°")^"'^ ore.' 

(a) the weight on the first factor of {TZ{GL„jUm) ® T^iGLm/Um))^"" should correspond 
to a partition (i.e., it should be a polynomial weight), and 

(b) the weight on the second factor should correspond to a partition with even parts. 

Remark: The content of Corollary 9.3 in terms of multiplicities is the Littlewood Restriction 
Formula [H], [IE]; see formula (2.4.1), see (5.7) with (4.19), ^7\] see Theorem 1.5.3 and 
2.3.1, and [Sni- With this result it is possible to compute a basis of the reciprocity 
algebra for (G'L„,0„) using (TE]; see second preprint of P^ . 



10. Tensor Products for 0„ 

According to Theorem 6.1, we can compute tensor products for the orthogonal group via 
the algebra 

(7'(M.,„)/J(J-+J)^- ® {V{Mr,,)/I{J:e))'''Y°" ■ 



Here the stable range is n > 2(m + i). Then we have 



Furthermore, 

^i^P^iZi)) = ^i^P?f ® 5pS'°^ ® (c" ® CO) 
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Since Jn,m - «5(sp2^°^) and Jn,i ^ we see that 

^Hn,m+l®S{C'^®C'). (10.1) 

Thus, using equation (10.1), we see that 

(P(M„,^)//(J-+J ® P(M„,,)//(J„+ ))^°" 

®{El^)f°- ® F>;^^,^ ® 5(C- ® CO 

^ (|0(^(n))''" ® j ® S{C- ® CO. 

Note that F^^-^ is the GL„ representation generated by the highest weight of the 0„ repre- 
sentation E^^-^ and both [F^n))^"^ (-^(n))*^" '-'^^ dimensionaL 
Hence, finally we get 



((^(M„,^)//(J-+J ® P(M„,,)//( Ji))^°") 



^iFtn)f" ® j ® 5(C- CO j . (10.2) 

We can interpret this algebra in term of tensor product algebras for general linear groups. 
According to Theorem 3.4(c) and (3.8), as a GLm x GLe module, we have 

5(C™®CO^0F(t)®4). (10.3) 
s 

We also know that 

n{GLjUm X GU/U,) ~ Fi;^^ (g) F(,y (10.4) 

Since this algebra is bigraded by ^ and z/, we can consider the "diagonal" (0, l)-subalgcbra 

An{GLjUm X GLi/Ui) ~ F(t) (g) Ff,^ (10.5) 

5 

resulting from requiring the two partitions to be the same. Evidently, the algebra S{C"^^C^) 
is isomorphic to A.TZ{GLjn/Ujn x GL^/Ue) as GLj^ x GL^-module. They are not isomorphic 
as algebras, since ATZ{GL^/Um x GL^/Ue) is graded by ^4+ x A^, while S{C^ ® C^) is not. 
However, we may filter iS(C'"®C^) by the representations of GL„i x GL^ (or of cither factor) 
using the dominance filtration (see §3.2), and then the associated graded algebra will be 
isomorphic to ATZ{GLm/Um x GLi/Ue) by Theorem 3.2: 

xA+('5(C'" ® CO) ^ An{GLmlUm x GLt/Ut). 
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Now turn to the first factor 0^ ^(-^(n))^" ® ^{m+e)j right hand side of equation 

(10.2). According to Theorem 4.1, we can write this as 

0(F(^„))^" ^ - (F^n) ® ® F^m) ® F^,y (10.6) 

A a,l3 

Combining equations (10.2), (10.3) and (10.6), we see that 

{{V{M^,m)/I{J:,J ® 7'(M„,,)/7( Ji)) °") 

© (^(n) ® <)) ® ® < ) ® ( © ® Fl, 

(\ UmXUe 
© i^in) ^ <)) ® (^M ® ^m) ® {F^e) ® 4)) (10-7) 
a,/3,<5 / 



© (^(n) ® <)) " ® (^M ® F^m^f"" ® (^(^) ® 



At this point, this is an isomorphism of graded vector spaces, not an algebra isomorphism. 

We may interpret the last expression in (10.7) analogously to (10.4) and (10.5). We have 
the (polynomial) tensor product algebras 

{n+{GL,/Uk) ® n+{GLk/Uk)f'^ ~ ® Fi^y' 

X,fj, 

for k = n,m and i. If wc form the tensor product of these, wc get 

(7^+(G'L„/C/„) (8) n-^{GLjUn)f- (8) {n+iGLjUj «) n+{GLjUm)f"^ 
{n+{GLe/Ue) ® n+{GLe/Ue)f' 

Let us denote this algebra by TTn^rn/- The algebra TTn^ra^i is (^n)^ ^ (^m)^ ^ (/l^)^-graded. 
If we require that A = a, or that n — /3, or that u — 5, then we obtain (0, l)-subalgebras of 
TFn,m,i- If 5 = tt, we will denote it by Ai 3ir„ and so forth. The subalgebra obtained by 
requiring that all three diagonal conditions occur at once will be denoted by using all three 
A's. Thus we will write 

= E (^(n) ® ^n)) ® (^M ® ^m) """^ ® {f'^^ ® F^^^Y' (10.8) 

a,l3,5 

We see from equations (10.7) and (10.8), that Ai_3A2,5A4^62r„_^_^ and 

UmxUe 



{{V{M^,m)/I{J:,J ® P(M„,,)//(J-+ ))^°") 
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are isomorphic as multigraded vector spaces. They may not be isomorphic as algebras, 
because 

[{v{M^,m)/i{j;tJ ® )) 

is not graded, while we see from equations (10.7) and (10.8), that Ai_3A2,5A4^6^n,m,£ is. 
However, if we pass to the associated graded of S{C"^ C^), then the two algebras do 
become isomorphic. We record this fact. 

Theorem 10.1. Assume the stable rangen > 2{m+i). We have the following isomorphisms 
of (A+)^ X (A+)^ X [A^Y -graded algebras: 

(/ Jl \ UmXUl\ 

[{V{M^,rn)/HJ;tJ^nMn,i)/I{J:,,)r'') j 

Remark: The content of Theorem 10.1 in terms of multiplicities can be found in see 
formula (2.1.2), |2H1; see Theorem 4.1 and jM!- 

11. Restriction from On+m to 0„ x Om 

Consider now branching from On+m to the product 0„ x Om- We look at the action of 
On+m. on Mn+m/ by multiplication on the left. Here we simply state the relationship of the 
reciprocity algebra for {On+m, On x Om) in the stable range, in relation to the GLn tensor 
product algebras. We shall omit the proof. 

The content of Theorem 11.1 in terms of multiplicities can be found in ^E]; see formula 
(2.2.2), [2H1; see (2.16) and [2Z1; see Theorem 2.5 and corollary 2.6. With this result it is 
possible to compute a basis of the reciprocity algebra for {On+m, On x Om) using see 
second preprint of [T3] . 

Theorem 11.1. Assume the stable range min (m, ra) > 2i. We have the following isomor- 
phisms of {Aj)^ X {Aj)^ X {A^)'^ -graded algebras: 

^'^(A+)3x(A+)3x(A+)3 {^{Mn+ml) / I{Jn+m,e)Y°"''^°"'''^' 

~ (7^+(GL,/^/,) ® n-^{GLe/Ue) ® 7^+'(GL,/f/,))''^ . 
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Appendix: A Proof of the Separation of Variables Theorem 

We provide a simple proof here for G = 0„. It could be adapted easily for Sp2n acting on 
copies of C^" or GLn acting on copies of and C"*. 

Let On act by the usual left multiplication on Mn,m, the n x m matrices. We assume that 
n > 2m, which includes the stable range n > 2m. Let Vij be the invariant pairing between 
the i-th column and the j-th column. Recall that J'n,m = 'P{Mn,m)'^" is generated freely by 
the homogeneous quadratic polynomials {vij} in this range (see Theorem 5.2.7). Also, 
the space of harmonics, i.e., polynomials annihilated by all the differential operators dual to 
the rjj's, is denoted by 7i„,m as in Theorem 3.3(a). 

Theorem (Separation of Variables) If n > 2m, then 

Remarks. Proofs of this result for orthogonal groups (see Theorem 2.5 of [^21) and for 
symplectic groups (see Theorem 1.10 of [HHl) are given by Ton-That using results of jSH]- 

Proof of Separation of Variables Theorem: Let I{Jnm) be the ideal in V{Mn^m) 
generated by rj^'s with zero constant terms, and consider P(M„ ^) as an I{Jnm) module by 
multiplication. 

We shall need the notion of a regular sequence. First, denote the ideal in P(M„ m) gen- 
erated by {/i, . . . , /J by the symbol < /i, . . . , /, > V{Mn,m)- A sequence {/i, . . . , fk} C 
I{Jnm) forms a regular sequence for V{Mn^m) if 

(a) fi is not a zero-divisor on V{Mn,m)/ < fi, ■ ■ ■ , fi-i > 'P(^n,m) for all i = 1, . . . , k, 
and 

(b) V{Mn,m)/ < /i, • • • , /fc > V{Mn,m) is uou-zero. 
Geometrically, saying that a given function / is not a zero-divisor on 

V{Mn,m)/ < fi, ■ ■ ■ , fi-i > is the same as saying that / does not vanish identically on any 
irreducible component of the zero set of {/i, . . . , This in turn is the same as saying that 

each irreducible component of {/i, . . . , /} has dimension one less that the component 
of the zero set of {/i, . . . , /j-i}. 

Separation of variables would follow from knowing that the Vi/s (in some order) form a 
regular sequence for V{Mn,m)- In fact, you can take any order you want. For an ideal / in 
a commutative ring S, we have the chain: 

and we can thus form the associated graded algebra 

GiiS = S/I © I/P © P/P © . . . 
with multiplication (setting S = I^) 

r/r+^ © P/P+^ — > ji+i/ji+i+i 

induced by multiplication on 5*. In our context, S = V{Mn,m) and V{Mn,m) / PJnm) is the 
coordinate ring of the null-cone and as a linear space (in particular, as a On x GLm module), 
it is isomorphic to 'Hn,m- If {^ij} C I{Jnm) is a regular sequence of V{Mn,m), then we have 
a nice presentation of Gi j^^j+^-^V (Mn^m)'- 
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Ree's Theorem (see Theorem 2.1 of "S^) /// is generated by a regular sequence /i, . . . , /„, 
then the map : {S/I)[xi, . . . ^ GrjS, sending Xi to the class fi in I /P is an isomor- 
phism. 

Ree's Theorem thus imphes that as vector spaces, we have S = S/ 1 ® C[/i, ...,/„], and 

in our context, V{Mn,m) = 'Hn,m ® Jn,m- 

Thus we want to show that indeed {vij} form a regular sequence in I{J^^. To show this, 
consider the map from M„ „^ to the m x m symmetric matrices iS^(C™') by putting the Vij in 
a matrix: 

Q : M„,^ 52(C") where Q{T) = T% T e Mn,m. 

First observe that this map is 0„ x GL^ equivariant: 

QigTh) = h'Q{T)h, geOn,he GL^. 

Further, the 0„ and GL^ actions commute. 

Let us study the fibers of the map Q. Define the following rank m matrices in Mn^m'- For 
A; = 0, 1, . . . , m, let 

Tk = \C\C2 ■ ■ ■ CfcC^-f 1 . . . Cm ] gm„ 

,m 

where {ci, C2, . . . , Ck} is an orthonormal set of non-isotropic vectors in C" and {ck+i, ■ ■ ■ , Cm} 
is an orthogonal set of isotropic vectors in C", such that each of the Cj with j > k are 
orthogonal to the Cj with j < k. It is easy to see that 

\lk ol 



QiTk 







where is the k x k identity matrix. Next, for i = 0,1,..., fc, define the sets 
6fc C Mn,m as follows: 

Gfc = {X G Mn,rn \ Q{X) = Q{Tk) and rankX = rankT^ = m}. 

Let us remind our readers on the following version of Witt's Theorem (see Theorem 3.7.1 of 



Witt's Theorem Given two nxm matrices Ti and T2, there is an orthogonal nxn matrix 
g such that gTi = T2 if and only if Q{Ti) = Q{T2) and kerTi = kerT2. 

Since X and Tk are of full rank, i.e., rankX = rankT^ = m , thus ker X = ker Tk = {0}. By 
Witt's Theorem, is an On orbit in the fiber Q^^{Q{Tk)) C Mn-m- The full rank condition 
gives the openness and denseness of this orbit. The null cone (or null fiber) NGQ = Qq 
corresponds to A; = 0. 

We claim that the fibers of the map Q are all varieties of the same dimension, i.e., Q is 
an equi- dimensional map: 

Proposition Consider the On x GLm- equivariant map Q. Then for each Y G iS^(C™'), 

(a) the fiber Q~^{Y) is an irreducible variety, invariant under On, and contains an open 
dense orbit of the form ■ h, for some h G GL^ and some k = 0,1, . . . ,m. 

(b) the dimension of each fiber Q^^{Y) is f = nm — Eiiz^+ll^ which is independent of the 
fiber. 

Remark: The inverse image of any symmetric matrix contains exactly one 0„ orbit 
consisting of invertible matrices. This is dense in the whole inverse image. It is the condition 
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that n > 2m which permits injective maps into isotropic subspaces (i.e., such that the pulled 
back form vanishes). 

\h 0^ 



Proof of Proposition: For Y E iS^(C'^), we can find h E GLm such that h^Yh 
for some k = 0,1, . . . ,m. 

If X E Mn,m is such that Q{X) = Y, then Q{Xh) = h^Q{X)h = h*Yh -- 





h 




Further, rankX/i = rankX. In other words, the fiber Q^^(Y) associated to y G 5^(C™) is 
the closure of an open dense orbit given by ■ h, and hence an irreducible variety. 

The fact that the orbits 0^ (and their translates) have the same dimension follows from the 
computation of the dimension of the pointwise stabilizer in 0„ of E Mn^m- The pointwise 
stabilizer can be easily computed for each k = 0,1, ... ,m. The dimension of each fiber is 
/ = nm — £]l!^±ll. (You can see this dimension from the null cone pretty easily because the 
ring of On invariants is freely generated by polynomials rj^'s.) □ 

Since the mapping Q has equi-dimensional fibres, if C iS^(C™) is any irreducible variety 
of dimension e, then Q~^{V) will be an irreducible variety of dimension e + /. The Q{rij)^s 
are coordinates on iS^(C'"), so the variety defined by d of them is a subspace of codimension 
d. It follows that the pullback of this subspace by Q is also irreducible and of codimension 
d. Therefore, the dimension of the zero set of r^j's decrease by 1 at each stage, making {vij} 
a regular sequence (see Lemma 4 on page 105 of [S31)- D 
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